1. Field of the Invention
This invention relates generally to the field of software testing and, more particularly, to the field of testing graphical user interface programs.
2. Description of the Related Art
One aspect of the field of software testing focuses on the testing of graphical user interface programs (GUI programs). A GUI program is a program that provides functionality which is not accessible from a command line but is accessible through the program's graphical user interface. Typically, GUI program testing takes place during a product development cycle of a new program. GUI program testing may also take place during the development of a new version of an existing GUI program.
GUI program testing often relies on a test engineer manually trying different user input combinations and noting anomalies as they occur. After the engineer changes the software to address the anomalies, the manual testing process is repeated until the test engineer verifies that the anomalies have been fixed. Manual testing has several drawbacks. For example, it is prone to error because it relies too heavily on human involvement in the testing process.
Due to the inadequacies of manual testing techniques, automated testing techniques have been developed. FIG. 1 illustrates a computer system 100 for testing a GUI program 101 using automated testing techniques found in the prior art. The computer system 100 includes a display device 103, a storage device 105, an input device 107, and a computer 109. The computer includes an interface 111, a processor 113, and a computer memory 115, which stores the GUI program 101. The GUI program includes a GUI component 119 and an engine component 121. The GUI component updates and retrieves information from a GUI 117. In short, the GUI component has program code to control changes to the GUI. Typically, the code is generated automatically from GUI builders. Engine code is code that implements the program's functionality. In other words, engine code generates the information to be displayed on the GUI.
Prior art automated testing methods simulate input events (such as mouse clicks) in order to test the same data processing paths that are invoked in response to user input. In this way all functionality in a GUI program is tested through the program's GUI component. The prior art methods begin the testing process by having a user enter data through a GUI 117 using the input device 107. For example, the inputs may indicate that a "File Open" command should be invoked. The method records the user inputs and saves them to a separate file. The recorded inputs are used as a test script 123 later in the testing process. A test script engine 124 drives the test script 123. In response to the inputs, the GUI 117 changes its state. For example, if the inputs indicate that a "File Open" operation should be invoked, then the method opens a file and displays it on the display device 103. In response to the changes in the GUI the method saves the new GUI state as a bitmap in a raster file (also called saving a "snapshot" of the GUI). The bitmap may also be saved in any other type of image format.
The snapshot acts as a baseline for comparing the program's response to user inputs at an earlier point in the development process with the program's response to the same user input at a later point in the development process. For example, as the development process continues, changes are made to the program code underlying the GUI component 119 and the engine component 121. In order to test the changes using automated techniques, the prior art method replays the user inputs recorded earlier. In response to replaying the user input, the GUI component 119 invokes the engine component 121 which in turn instructs the GUI component to redraw the GUI. In this way the engine component 121 is invoked through the GUI component 119. The method then takes a snapshot of the changes to the GUI 117. Finally, the earlier snapshot is compared to the later snapshot. If the snapshots are identical then the method concludes that the changes made to the program work properly. Non-identical snapshots may indicate that a bug has been introduced into the program.
There are several disadvantages associated with the automated testing techniques used in the prior art. The maintainability of the "test scripts" (i.e., the recorded user inputs) is low. In other words, as the development cycle progresses, the GUI 117 typically undergoes a wide variety of changes. Therefore, the user inputs made at an early stage of development may be meaningless at a later stage of development. One example could involve recording a user selecting a pull-down menu on a first screen and then invoking a "File Open" command from the pull-down menu. If the "File Open" command was removed from the pull-down menu of the first screen during the development cycle then the test script would be invalid when testing the revised pull-down menu. In this case a comparison of GUI snapshots would be impossible.
Another disadvantage associated with the prior art method involves the difficulty involved in porting the test scripts to different platforms. If the "baseline" snapshot is taken on a first platform and subsequently compared to a snapshot taken on a second platform then the bitmaps will not match if any of the system parameters change between the platforms. For example, snapshots taken on an Intel.TM. platform are often more condensed than snapshots taken on a Sparc.TM. platform. Therefore, the snapshots will not match even if the test script causes the same actions to occur to the GUIs of both platforms.
Another disadvantage associated with the method of the prior art is the amount of storage space required to store the snapshots. Each snapshot requires a significant amount of storage space. Often, over 300 snapshots are saved in order to fully test one GUI program. Such storage requirements are expensive and therefore provide a disincentive to using the prior art method.
Finally, even if the prior art testing method concludes that a bug was introduced into the GUI program, it is still difficult to determine whether the bug is in the GUI component 119, or the engine component 121, or some combination of the two components.